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RELIABILITY  ESTIMATION  UNDER  PLAUSIBLE  ASSUMPTIONS 


by 

Frank  Prosch&n 


1.  Introduction  and  Summary  Most  analyses  of  reliability 
problems  assume  that  the  form  of  the  underlying  failure  distributions ) 
is  known;  the  parameter  may  be  assumed  known  or  unknown.  Families  of 
failure  distributions  commonly  used  are  the  exponential,  gamma,  normal, 
and  lognormal. 

The  weakness  of  such  analyses  is  that  the  conclusions  reached  may 
be  grossly  in  error  if  the  assumption  as  to  underlying  failure  distri¬ 
butions  is  incorrect.  In  fact,  the  error  in  the  original  assumption 
may  be  greatly  compounded  in  arriving  at  the  final  conclusion,  especially 
if  the  conclusion  concerns  a  tail  probability  or  if  the  system  analyzed 
is  complex. 

In  this  paper  we  present  an  expository  survey  of  recent  research 
in  reliability  estimation  based  on  assumptions  made  not  singly  for 
mathematical  convenience  but  because  they  correspond  to  the  physical 
situation.  The  class  of  statistical  problems  considered  thus  lies 
somewhere  between  parametric  problems  (in  which  the  underlying  failure 
distributions  are  assumed  known  up  to  a  finite  number  of  parameters) 
and  nonparametric  problems  (in  which  do  information  is  assumed  available 
concerning  the  underlying  failure  distributions). 

In  Section  2  we  summarize  research  concerning  the  errors  resulting 
from  using  standard  exponential  life  test  and  estimation  procedures 


when  in  fact  the  distribution  has  an  increasing  failure  rate  or  a  decreasing 
failure  rate.  This  raises  the  natural  question:  Given  a  sample  from  an 
unknown  distribution,  does  the  distribution  have  an  increasing  failure 
rate  (or  alternately  decreasing  failure  rate)?  In  Section  3  we  summarise 
research  on  tests  for  increasing  (decreasing)  failure  rate.  Assuming  then 
that  the  assumption  of  an  increasing  (decreasing)  failure  rate  is  plausible, 
the  next  problem,  discussed  in  Section  4,  is  the  estimation  of  the 
failure  rate  function,  the  distribution,  and  the  density,  A  different 
class  of  problems  is  discussed  in  Section  5>  namely  the  estimation  of  an 
evolving  reliability,  based  on  observations  obtained  at  successive  stages 
of  evolution.  Unlike  the  usual  treatment  in  the  literature,  no  assumption 
is  made  concerning  the  mathematical  form  of  the  reliability  growth; 
rather,  only  monotonicity  of  reliability  in  succeeding  stages  is  assumed. 
Maximum  likelihood  estimators  and  conservative  confidence  bounds  are 
obtained  for  parameters  or  functions  arising  in  such  reliability  growth 
problems. 

In  order  to  present  precisely  the  results  of  the  succeeding 
sections  we  introduce  the  following  definitions  and  notation.  Let  F 
he  a  right  continuous  distribution  such  that  F(0  )  -  0  .  If  F  has 

f  (t) 

a  density  f  then  r(t)  =  ZT~J~  is  known  as  the  failure  rate,  where 

F(t) 

F(t)  =  1  -  F(t)  represents  the  survival  probability  or  reliability. 
Physically,  ^r(t)dt  is  the  conditional  probability  of  failure  in  (t  , 
t+dt)  given  survival  until  time  t  .  We  may  verify  by  differentiation 
that  under  the  assumption  of  increasing  (decreasing)  failure  rate 
log  F(t)  is  concave  (convex)  on  [0,«)  .  More  generally,  whether  a  density 
exists  or  not,  we  shall  say  a  distribution  has  an  increasing  failure 
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rate  (IFR)  if  log  F(t)  Is  concave  on  [0,«)  }  and  a  decreasing 
failure  rate  (XJFR)  if  log  F(t)  is  convex  on  (0,«).  Properties  of 
IFR  and  DFR  distributions  are  discussed  in  Barlov,  Marshall ,  and 
Proschan  (1963).  Applications  of  such  distributions  to  reliability 
problems  are  presented  in  Barlov  and  Proschan  (1965). 


»sv&Qsp&&*$ssa£!i!>‘ 
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2.  Errors  in  Using  Exponential  Procedures  When  the  Distribution 
Has  Monotone  Failure  Hate 


This  section  is  "based  on  Barlow  and  Proschan  (196IO.  We  give 
proofs  initially  to  convey  the  type  of  mathematical  argument  involved; 
after  that  we  refer  the  reader  to  the  source  paper  for  detailed  proofs. 

2.1  Censored  Sampling  Without  Replacement  Assume  n  items  arc 
put  on  life  test  simultaneously  at  time  0  .  Let  X_^ 

denote  the  ordered  observations  corresponding  to  successive  failures. 

If  the  failure  distribution  has  exponential  density 


(2.1) 

/  \  1  ~x/0 
g(x;0)  -  e  ,  x  ^  0 

,  G>  0  , 

then 

r  ^ 

r 

(2.2) 

3  =  M  )  X  +  (n-r)X 

rn  r  {.  i  r . 

j~  =  ~  ^  (n-i+l)(X^-X^_^)  ,  1  C  r  £  n  , 

i=l 

i~l 

is  the  maximum  likelihood  (ML)  ,  minimum  variance  unbiased  (MVU) 
estimator  for  6  based  on ‘the  first  r  order  statistics.  (Epstein 
and  Sobel,  19530  We  shall  show  in  Theorem  2.2  below  that  this  estimator 
is  biased  high  (low)  when  the  underlying  failure  distribution  is  IFR  (DPR). 
We  will  need 

Lemma  2.1  Let  X^  Xn  be  order  statistics  from  F  , 

IFR  (DPR).  Then  the  normalized  spacings  (n-i+l)(X^  -  X^_^)  8X6 

stochastically  decreasing  (increasing)  in  i  . 

Proof  Assume  F  is  IFR  .  Then 

PtnXj^  i  x]  =  (F(|))n  (F^))11'1 

l/t  ■— 

since  (F(t)}  '  is  decreasing  in  t  by  the  log  concavity  of  F  . 


Let  F  (x)  =  -1— ^  conditional  distribution  of  an  item  of 

*(u) 

age  u  .  Then  since  F  is  IFF,  F  (x)  >  F(x)  .  Given  that  X  =  u 

is  observed,  X^-X^  ^-^ributed  as  the  first  order  statistic  from 

a  sample  of  size  n-1  each  with  distribution  F  (x)  .  Hence 

u 

PKn-DGg-^)  i  x  I  ^  =  tt]  =  (Fu(^-)}n'1  . 


Conditioning  on  ,  we  have 

Ptn^  J,i].  (F(|)}n  j.  (Fu(^r))n‘1=  P[(n-])(X2-XL)  x  1 X,-  u]  ,  u  ;>  0 
Unconditioning, 

PtnXj^  ^  x]  ^  J  (Fu(^~)}n_1  dG(u)  =  PtCn-lJfXg-^)  ^  x]  , 


where  G(u)  =  1-  [F(u)}n  is  the  distribution  of  X^  . 

Thus  we  have  shown  that  nX^  is  stochastically  larger  than 
(n-l)(Xg-X^)  .  In  a  similar  manner  we  can  show  that  (n-i+l) (X^-X^ 
is  stochastically  larger  than.  (n-iHx^-X^)  for  i  =  2,3,...,n  . 

All.  inequalities  are  reversed  for  DFR  distributions.  |j 
Now  we  can  show 

A 

Theorem  2.2  a)  If  F  is  IFF  with  mean  9  .  then  9  <  E  9  < 

-  )  rn  ^  r 

A 

for  r=d,2,.  ..n.  b)  If  F  is  DFF  with  mean  9  ,  then  0  <£_  E9  £  9 
for  1  r  <£  n  .  All  inequalities  are  sharp. 

Proof  a)  From  Barlow  and  Proschan  (1965),  p.  33,  we  know 


n9 


that  E9,  >  9  .  Also 

In  ^ 


h(r)  =  )  1E(n-i+l)(X 


i'W  -  6} 


i=l 


exhibits  at  most  one  sign  change  as  a  function  of  r  since  E(n-i+l)  (X^X^ 
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is  decreasing  in  i  by  Lemma  2. 1.  Bit-  h(l }  \  0  and  h(n)  =  0  , 

A 

so  that  h{r)  ^  0  for  r  =  1,2,..., n  .  Hence  E9rn  ^  ©  •  Clearly 

the  bound  is  attained  by  the  exponential  distribution  so  that  it  is  sharp,. 

r  n 

To  show  the  upper  bound, note  that  2  X  +  (n-r)X  <[_  2  X  for  every 

~  i=i  1  r  i=l  1 

sample  realization.  Hence  Er9^n  <1  a®  >  so  that 

~  nQ 

JJQ  /  _ 

rn  ^  r  .  Since  equality  is  attained  with  distributions  degenerate 
a,t  9  (which  is  the  limit  of  IFR  distributions)  the  bound  is  sharp. 

For  the  proof  of  b),  see  Barlow  and  Proschan  (196k),  Corollary  2.3.  || 


2.2  Censored  Sampling  with  Replacement  Suppose  now  that  failed 
items  are  replaced  at  failure.  In  this  case  the  bias  of  the  usual 
exponential  estimate  for  9  is  even  greater  than  in  the  non-replacement 
case. 


*  'th 

Let  X.  denote  the  time  of  the  i  failure  when  failed  items 
i 

are  replaced.  Then  under  the  exponential  assumption,  the  ML,  MVU 

✓w  nX* 

*  x 

estimate  of  9  based  on  the  first  r  failure  times  is  9  =  - . 

rn  r 

Then  Barlow  and  Proschan  (1964)  prove 

Theorem  2. 3  If  the  underlying  failure  distribution  F  is  IFR 

with  mean  9  ,  then 

9  <_  E9  <(E9*  <~EX  ,  1  <  r  ^  n  . 

rn  ^  rn  •*  r  r 


2.3  Inverse  Binomial  Sampling  N&dler  (i960)  has  considered 
the.  following  type  of  sampling.  An  item  having  life  distribution  F 
with  mean  9  is  put  on  test  until  it  fails  or  reaches  age  t  ,  whichever 
occurs  first.  At  this  time  the  item  is  replaced  by  a  fresh  item. 

This  procedure  is  repeated  sequentially  until  r  actual  failures  are 
observed.  The  number  N  of  items  that  have  to  be  tested  until  the 
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r  actual  failures  are  obtained  is  a  random  variable.  Nadler  (i960) 

_x/0 

showed  that  when  F(x)  =  1-e  '  ,  an  unbiased  estimate  of  9  is 

®r(t)  =r{IXi+  (N-r)t} 
i=i 

where  the  X^,...,Xr  are  the  r  life  lengths  not  exceeding  t  . 

Following  the  proof  in  Barlow  and  Proschan  (196U),  we  show 

A 

Theorem  2.4  If  F  is  IFR  (DFR)  with  mean  9  ,  then  E9r(t)^  (^)  Q 

Proof  Let  F  be  IFR,  Z^  denote  test  time  elapsed 

st  th 

between  the  i-1  failure  time  and  the  i  failure  time,  i=l,2,...,r, 
th 

where  the  0  failure  time  is  defined  to  be  0  .  Then 


r 


i  =4 


Next  consider  an  alternative  testing  procedure  differing  in  that 

replacement  occurs  only  upon  failure.  Let  Z^  =  test  time  elapsed 
st  th 

between  the  i-1  failure  and  the  i  failure  under  the  alternate 

testing  procedure.  Now  since  F  is  IFR,  Z±  is  stochastically  larger 
than  Z^  .  It  follows  that 


r  r 


i=l  i— 1 


The  inequality  is  reversed  when  F  is  DFR  .  || 

Additional  results  are  obtained  in  Barlow  and  Proschan  (1964) 
for  other  sampling  procedures  which  illustrate  the  general  thesis  that 
the  use  of  exponential  life  test  and  estimation  procedures  when  the 
underlying  distribution  is  IFR  (DFR)  yields  estimates  of  mean  life 
which  are  biased  high  (low)  . 
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3.  Tests  for  Increasing  (Decreasing)  Failure  Rate 

3.1  The  Test  It  is  apparent  from  Section  2,  that  ve  should  he 
cautious  about  the  use  of  exponential  life  test  procedures  if  we  suspect 
that  the  underlying  failure  distribution  may  be  IFR  (DFR).  Thus  it  is 
appropriate  now  to  consider  tests  for  constant  vs.  increasing  (decreasing) 
failure  rate.  In  this  section  we  summarize  results  obtained  in  Eros chan 
and  Fyke  (1965). 

Let  Xg  . . .  £  X^  be  an  ordered,  sample  from  the 

distribution  F  ,  with  density  f  ,  where  f(t)  =  0  for  t  <  0  ,  and 
failure  rate  r(t)  .  We  wish  to  choose  between  the  following: 

Null  Hypothesis .  H  :  r(t)  =  X  ,  X  an  unknown  positive  constant. 
Alternative  Sypothesis.  r(t)  is  strictly  increasing. 

The  (nonparametric )  test  statistic  is  computed  as  follows.  Let 

D*  =  X,  ,  D*  =  X  -X  ,. ..  D*  =  X  -X  .  ,  the  spacings,  and  D  »  nD* 

1  1  2  2  1*  ’  n  n  n-1  ’  ’  1  1 

D_  =  (n-l).D*  . ..  ,D  =  D*  .  the  normalized  spacings.  Let 

2  2  n  n 

fl  if  D.  >  D  for  i,  j=l,2,...,n 
V  =  1  1 

^  0  otherwise 


The  test  statistic  is 

n 

r-i 

(3.1)  \ 

L  Vij  * 
i,j=l 

1  <  d 

We  reject  the  null  hypothesis  at  the  a  level  of  significance  if 

V  >  v  ,  where  v  is  determined  so  that  P[V  >  v  j  H  ]  =  a  . 
n  '  na  n a  nr-  na  0 

Heuristically  we  may  justify  the  test  as  follows.  Under  the  null 
hypothesis,  ,  D2,*'*,I>n  arS  indePsndently  distributed,  each  with 
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-to 


density  to  ,  as  shown  in  Epstein  and  Sobel  (l955)>  so  that  the  D^ 


are  stochastically  identical  and  therefore  the  are  equally  likely 

to  be  0  or  1  .  However,  under  the  alternative  hypothesis,  each  , 
and  consequently  ,  tends  to  be  larger,  so  that  rejection  of  the  null 


hypothesis  occurs  for  large  values  of  V 


n 


Since  under  the  null 


hypothesis,  the  distribution  of  V  is  known,  we  have  available  v 


na 


3. 2  Distribution  Under  the  Null  Hypothesis  Under  HQ  all 
orderings  of  D^,...,S^  are  equally  likely.  Thus  we  may  use  the  results 
obtained  by  Kendall  (1958)  and  Mann  (1945 )  for  the  number  of  orderings 
of  ...  ,D^  with  exactly  k  inversions  of  indices;  an  inversion  of 
indices  i  <  j  occurs  when  D  ^  D,  .  Both  Mann  and  Kendall  provide 

1  J 

tables  for  p[v  <  k]  ,  n  <  10  .  For  n  >  10  ,  we  may  approximate 
n 

P[Vn<;k]  closely  using  the  fact  that  V,  Is  asymptotically  aormal 
with  mean  and  variance  0^  »  _  Thus 

is  available  for  given  a  . 


na 


3.3  Unbiasedness  of  Test  Following  Proschan  and  Fyke  (1965 )  it 

is  easy  to  show  that  the  test  statistic  Vn  is  unbiased,  i. e. , 

P[Vn  vm  I  a  for  0  <  a  <_  1  ,  n  «  2, 3, . . .  . 

To  prove  unbiasedness,  make  the  transformation  X^  =  -log  F(x^)  . 

If  Z^(Z^)  ,  i  =  l,...,n  ,  are  the  unordered  X^(X^)  ,  i  =  l,...,n  ,  then 

P[Z^  >  u]  =  P[log  F(Z  )  <  -u]  =  P[F(Zi)  <  e“U]  =  e~U  .  Thus  each  Zj 

is  distributed  according  to  the  exponential  distribution  with  unit  mean. 

Moreover  since  the  are  independent,  so  are  the  Z^,...}Z'^  . 

Next  let  =  (n-i+l)(X^  -  )  i  =  l,...,n  ,  where  X^  =  0  . 

Now  note  that  X^  is  a  convex  increasing  function  of  X^  which  is  0 

for  X  =  0  .  It  follows  that  D’  D*.  implies  D  D.  for  i  <  j,i, 

i  ^  J  i  j 


-9- 


j=i,...,n  .  Thus  V  ^  ,  i  <  j,l,  j*JL, . . .  ,n  ,  where  =  1 

if  D'  >  D'.  .  Hence  V  >  V'  where  V'  =  Z  V  so  that 

i  ^  J  n  ^  n  *  n  ^  i  j 

P[Vn  Vjq  I  >.  a  for  0  <  a  ^  1  ,  n  =  2,3, . . .,  by  Lehmann  (1959, P. 73). 

3.4  Asymptotic  Distribution  Under  Alternative  Hypothesis  Froseh&n 

and  Pyke  (1965)  prove  that  the  statistic  Vq  when  suitably  normalized 

is  asymptotically  normally  distributed  for  a  wide  class  of  alternatives. 

Theorem  3.1  Let  the  distribution  F  be  an  absolutely  continuous 

1FR  with  failure  rate  function  r  .  Let  p  exist  and  be  continuous 

a.e.  on  (0,l)  ,  where  p(u)  =  l/r(F  ^(u))  ,  the  reciprocal  of  the  failure 

rate  after  it  is  transformed  by  means  of  F  ^  onto  the  unit  interval. 
~3/2 

Then  n  J  (V  -  u.)  converges  in  law  as  n  ->  w  to  a  normal  random 

o 

variable  with  mean  0  and  variance  0~  ,  where 

pi  pi  , 

(3.2)  M-  *  /  /  p(u)[p(u)  +  p{v)]  dv  du  , 

,J0‘Ju 


(5.3) 


2  2  .2 
0  =  0„  +  20  -T  C  , 

5  SB  R  ' 


2 

0  3 
LS 


f  p  1  p  V  p  V  pi  pi  pi  pi  r*l  p  V)  p 

j ill  +  /  /  /  -2  /  /  /  jlo(u)p(w)p  (v)[{p(u)+p(v))[p(v)+p(w)) 

u  0J  O'"*  0  0  0J  v^  v  0  n'",  ’■  n  J 


0^  v'-’  0 


fp(u)p(w)  +  p(u)p(v)  +  p(v)p(v)}3  “du  dw  dv  , 


n  1  p  1  g 

0c.p~  /  /  p(tt)p(v)[p(u)  +  p(v)j  [p(u)  -  B(v)]dv  du  , 

SFl  J0Ju 


2  f 1  2 

*  /  £ 


(u)du  ,  and 


^(v^CL-v)"1!  p  p’(u)(l -u)J  p(v)[p(u)+p(v)]"2dv  6n-J  p'(v)(L -v)J  p(i)[p(u)+p(v)]‘ 


du  dv) 
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5  AHvrrototic  Rel  ative  Eff ieianev  To  conmare  the  V  test 

with  other  possible  tests  wa  use  the  criterion  of  asymptotic  relative 
efficiency  (ARE).  For  some  specified  set  of  alternatives  indexed  by 
0  ,  the  ARE  of  one  sequence  of  tests  based  on  a  sequence  of  asymptotically 
normal  test  statistics  {T,}  against  a  second  sequence  of  tests  based 
on  the  asymptotically  normal  test  statistics  {t^}  is  defined  as 


(5.4) 


l  Ci4(60)]2/o|(90))C  En4-(e0  >  3  <o0>  3 -1 


2 

whenever  it  exists.  In  (3.2),  ^(9)  an<*  (7^,(9)  denote  the  limiting 
mean  and  variance  respectively  of  (Tn)  ,  p.,J,  denotes  the  derivative  of 
with  respect  to  9  and  9^  denotes  the  null  hypothesis.  A  similar 
interpretation  is  understood  relative  to  T  . 

(a)  Likelihood-ratio  test  for  Weibull  alternatives  Suppose  that 
Hq  is  as  before,  but  is  specialized  to  the  case  in  which  the  underlying 

distribution  is  the  Weibull  with  increasing  failure  rate: 


F(x)  =  1-e 


,  A>  0  ,  9  >  1  ,  x^O, 


where  Tv  assumed  known.  We  may  readily  verify  that  the  likelihood  ratio 

n 

test  is  to  reject  H  if  TW  >  c  ,  with  TV  =  (l-AX  )log  X  . 

0  n  x  a  n  i=1  ni  ni 

Also,  we  may  calculate  for  the  derivative  of  the  limiting  mean  and  for  the 

limiting  variance  of  T  ,  p.'  (l)=(in  A  +  r  -  l)^  +  -§•  =  0  (l),  where 

n  Tw  o  To 

T  =  .577215...,  so  that  {|i*  (l)  2/a2w(l)  »  (in  A+  T  -  l)2  +  j'  . 

T  T 

For  ,  using  (3.2)  and  (3.3),  after  some  calculation  we  find 


(p.’(l))2/cj2(i)  =  2Y 


Thus  the  ARE  of  V  with  respect  to  T  reduces  to 

n  n 

ARE  =  . 5 -  .  As  A  -  0  or  «  ,  ARE  -  0 

W  (in  A-.  4228)+!.  6^9) 
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It 


all  A  >  0  ,  AHE^  £  » ^571  i  equality  is  attained  for  in  A  *»  .  4228  . 

(b)  Likelihood-ratio  test  for  Ganna  alternatives  Next  assume 
1  "  1  1  -  1  ■  “  ■  . . 

that  is  specialized  to  the  case  in  which  the  underlying  distribution 

is  the  Gamma  with  increasing  failure  rate.  The  density  is 

9  o-l  -Ax 

fM  “  Xr(9)  *  • 

The  likelihood  ratio  test  i3  to  reject  if 

is 

n 

Tn=  Zl08xni>ca  • 

i=i 

Tty  calculation  similar  to  that  in  (a),  we  obtain  for  the  ARE  of 

n 

V  relative  to  T  : 
n  n 

ARE-,  =■  .2040  , 

tr 

independent  of  A  . 

In  a  similar  fashion,  Proschan  and  Pyke  (1965)  compute  the  ARE 
of  Vn  relative  to  the  likelihood  ratio  test  against  the  Gamma  when 
the  true  distribution  is  the  Weibull,  and  the  ARE  of  V  relative  to  the 
likelihood  ratio  test  against  the  Weibull  when  the  true  distribution 
is  the  Gamma.  For  certain  values  of  the  parameters  in  each  case,  the 
ARE  is  >  1  ,  implying  that  the  Vn  test  is  better. 

Other  nonparametric  tests  for  IFR  (DFR)  are  possible.  For  example, 
the  coefficient  of  variation  under  is  1,  and  <£  1  under  . 

Moreover,  as  shown  in  a  forthcoming  paper  by  Marshall,  Olkin,  and 
Proschan,  the  sample  coefficient  of  variation  is  stochastically  smaller 
under  than  under  Eq  .  Thus  an  unbiased  test  of  vs. 

may  be  constructed  using  as  test  statistic  the  sample  coefficient  of 
variation. 
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4.  Maximum  Likelihood  Estimation  for  Distributions  with 
Monotone  Failure  Rate 


Suppose  now  we  have  established  either  from  physical  considerations 
or  from  tests  on  observed  data  that  the  underlying  failure  distribution 
is  IFR  or  DFR.  How  do  we  estimate  the  density  f  ,  distribution  F  , 
or  failure  rate  function  r  ? 

Grenander  (1956)  derives  the  maximum  likelihood  estimator  (MLS) 
assuming  F  is  IFR  .  Marshall  and  Pros  chan  (1965)  present  an  alternate 
derivation  of  the  MLE  in  the  IFR  case  and  also  obtain  a  MLS  for  the  DFR 
case.  They  show  that  these  estimators  are  consistent. 

4.1  Derivation  of  MLE  Let  X. .  ^ .  X^  £  . . .  £  be  a  sample 
of  n  ordered  observations  from  F  ,  IFR  .  Using  the  fact  that 

-/  r(z)dz 

(4.1)  F(x)  =  e  _c° 


we  nay  express  the  log  likelihood  L  as 

n  n 


n  x. 


(4.2) 


L  =  y  log  f(xi)  =£log  r(X±)  -  Yjf  T^dz  • 

i=l  i-1  I«L 


Without  further  constraint,  L  can  be  made  arbitrarily  large  by 
taking  f  (X^)  correspondingly  large.  Consequently,  we  first  consider 
the  class  3  of  IFR  distributions  with  failure  rate  bounded  by  M  . 

Aw  _ 

We  shall  first  find  the  unique  distribution  Jr  in  3"  for  which. 

n 

(4.2)  is  maximized.  Then  letting  M  -*•  00  we  shall  find  F^ 

n 

A 

converges  in  distribution  to  an  estimator  Fq  which  we  call  the  MLE 
for  F  in  ?  ,  the  class  of  IFR  distributions. 
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n 


As  shcvn  by  Grenander  (1956),  L  is  maximized  over  3-  by 
a  distribution  with  failure  rate  constant  between  observations. 
(Beuristically,  this  is  apparent,  since  on  the  one  hand  the  failure  rate 
cannot  decrease  between  observations,  and  on  the  other  hand,  any  increase 
in  failure  rate  between  a  pair  of  observations  can  only  result  in 
diminishing  the  possible  values  of  the  density  at  the  remaining 
observations.)  Thus  we  may  replace  L  by  the  function 
n  n-1 

(4.3)  Llog  • 

i=l  i=d 


The  maximization  of  (4.  5)  subject  to  r(X^)  <^...^r(Xn)  =  M  is 
performed  by  Grenander  (1956);  it  can  also  be  performed  as  a  direct 
application  of  Brunk  (1958),  Corollary  2.1  and  the  discussion  following 
(see  also  van  Eeden  (1956,  1957))*  This  yields  for  r  (corresponding 
to  F  in  *">  the  estimator 

(4.4)  r"(X :  )  =  min  (min  max  {  [r"1  +...+  r"1  ])_1,M)  , 

n  1  v£L+l  n<i  u  u  ' ~± 


where  r  =  M  and 
n 


(4.5)  =  C(n-j)(X  -X  )]  1  for  j=l,2, . . .  ,n~l  . 


M 

The  estimator  r  given  in  (4.4)  differs  in  form  from  the  one  given 
'  n 

in  Grenander  (1956)  but  is  equivalent  to  it. 

The  maximization  whuch  yields  (4.4)  may  be  described  as  follows. 
First,  find  the  maximum  cf  (4.4)  obtaining  (4.5).  If  there  is  a 
reversal,  say  r±  >  r  ,-*enset.  r(X±)=  r(xi+1)  in  (4. 5)  and  repeat 
the  procedure.  After  at  most  n  steps  of  this  kind,  a  monotone 
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estimator  is  obtained.  The  maximum  derived  vith  r(X. )  =  r(X  ) 

x  X4-JL 

can  be  directly  obtained  by  replacing  r.  and  r  by  their  harmonic 

1  1-fJL 

ri /  -i  -i  ..-i 
mean,  +r4_L,)j 


'i+1* 


Succeeding  steps  amount  to  further  such 
averaging  which  is  extended  just  to  the  point  necessary  to  eliminate 
all  reversals.  It  can  be  seen  that  this  is  exactly  vbat  is  called  frr 
in  (4.4)  taking  into  account  r(x)  M  .  (in  this  connection  see  also 
Ayer,  Brunk,  Ewing,  Reid,  and  Silverman  (1955)  and  Brunk  (1955). )  The 


'"M 

resulting  estimator  rn  is  of  the  form 


r^  =  min(r 


<\ 


Ei+1'ni+l 


,M), 


X  /!<  X 


V1 

*i-Xn  ’ 


h.  _+l 

i+l 


Where  £  *^+1^  ^  *•*  ^  \+l,n-l  '  0  “  nQ  <  \  <•  •  •<  \  <  > 


V 


and  r  ..  is  the  harmonic  mean  of  r 

°i+1’ni+i 


n,+V  n,+2>'~-  n 


The 


i+1 


n^  are  determined  by  the  rule  which  determines  the  extent  of  the  averaging. 

The  estimator  for  r  corresponding  to  F  in  "7  is  obtained  by 
letting  M -«•  os  in  (4. 4),  and  is  given  by 

(4.6)  r  (X  )  =  min  max  [v-u][(n-u)(X  -X  )  +...+  (n-v+l)(X  -X  J]’1, 
n  1  -v^i+1  u£L  U+1  u  v  v*x 


i=l,2,. . . ,n-l  and  r  (X  )  =  oo  .  For  the  remaining  values  of  x  , 

A 

r  (x)  is  0  for  0  <*  x  <•  X  ,  eo  for  x  >  X  ,  and  constant  between 
n  x  1  n  ’ 

A  /V 

observations.  The  corresponding  estimators  F  and  f  for  F  and  f 


n 


are  obtained  from  r  using  (4.1)  and  the  relation  f  (x)  =»  r  (x)F  (x)  . 

n  n  n  n 

4.2  Consistency  In  Marshall  and  Rroschan  (1965),  the  consistency 

of  the  MLE  of  F  ,  f  ,  and  r  are  proven.  The  results  may  be  summarized 


as  follows: 
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Theorem  4.1  If  r  is  increasing,  then  for  every  t  , 

0*.7)  r{t“)  £  lim  inf  rn(tQ)  £  11m  sup  rn(tQ)  £  r(t*) 

with  probability  one, 

A 

Corollary  4,2  If  r  is  increasing,  then  for  all  t  ,  lim  F  (t)=?F(t) 

n-w 

with  probability  one. 

Corollary  4,5  If  r  is  increasing  and  continuous  on  [a,b]  ,  then 

(i)  lim  sup  |  r  (t)  -  r(t)  |  =  0  , 
n-K»  a£t<J>  n 

(ii)  lim  sup  |  F  (t)  -  F(t)  |-  0  , 

n+oo  -oo^t<«) 

(iii)  lim  sup  j  f  (t)  -  f (t)  J  =  0  , 
nt-*oo  a£t<b  n 


each  with  probability  one. 

/V 

4.5  Additiv.  ia.1  Results  With  respect  to  a  certain  metric  rn(t) 
is  closer  to  r(t)  than  is  rn(t)  >  where 


0 

for 

0  1 1  <  \ 

r  (t)  a  < 
n 

C(n-j)(V 

V1 

for 

Xj  ^  t  <  Xj+1’  *  • ,n"1 

00 

for 

x  c  t  <  «  . 

n  ^ 

Note  that  2*  (t)  represents  the  "unaveraged"  estimate  of  the  failure 
n 

rate,  i.  e  ,  the  estimate  that  does  not  take  into  account  the  requirement 
that  r(t)  be  Increasing.  Let  Fn(t)  be  the  usual  eng>irical  distribution. 
Then  the  following  inequality,  a  special  case  of  the  results  of  Brunk  (1961 ) , 
is  obtained  directly  in  Marshall  and  Proschan  (1965). 


Theorem  4. 4  With  probability  one, 

f  {r  (t)  -  r(t)}2d?  (t)  ^  (t)-r(t)}2dP  (t)+rP{r  (t)-?  (t))2dFn(t)  . 

t/n  nu  n  n  j  u  n  n 

—00  "OO  ’  —00 

In  addition,  Barlow  and  Pros chan  (1965 )  obtain  the  MLE's  of  r  , 
f  ,  and  F  for  F  DFR  and  show  the  consistency  of  these  estimators. 
Finally,  they  discuss  maximum  likelihood  estimation  in  the  case  of  discrete 
IFR  and  DFR  distributions. 
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Estimation  of  Reliability  Growth 


An  important  group  of  reliability  problems  consists  of  those  in 
vhich  the  reliability  of  an  evolving  system,  is  to  be  estimated  at 
successive  stages  of  its  evolution.  Most  studies  in  the  literature 
have  assumed  a  priori  knowledge  of  the  form  of  the  function  governing 
reliability  growth.  (See  for  example ,  Lloyd  and  Lipow  (1962),  Chapter 
11,  Wolman  (1963),  and  Corcoran,  Weingarten,  and  Zehna  (196*0. ) 

Unfortunately ,  in  many  eases  the  only  a  priori  knowledge  actually 
available  is  that  the  reliability  at  successive  stages  of  evolution  is 
monotonicslly  increasing.  In  the  present  section  we  shall  show  how  to 
obtain  a  maximum  likelihood  estimator  and  establish  conservative 
confidence  bounds  for  reliability  assuming  monotonicity  in  reliability 
at  successive  stages.  A  more  detailed  treatment  will  appear  in  forth¬ 
coming  papers  by  Barlow,  Proschan,  and  Scheuer. 

5.1  Maximum  Likelihood  Estimation  of  JReliability  Irswth  A  system 
is  being  improved  at  successive  stages  of  development,  corresponding, 
say,  to  basic  design  changes.  At  stage  i  the  failure  distribution  of 
system  life  length  is  F_^  j  no  assumption  is  made  as  to  the  form  of 

F  , ...,F  or  the  relationship  among  them  except  that  F  (t)  ^>.  F  (t £>„.</> F  (t) 
lk  12k 

for  ail  t  ';>  0  .  Independent  life  length  observations 

are  obtained  at  stage  i  ,  i=l,2,...,k  .  From  these  we  wish  to  estimate 
Fx,F2,...,Fk  ,  especially  Ffc  . 

To  obtain  the  MLE  of  F^(t)  ,  0  £  t  <  »  f  i-l,...,k  ,  we  use  the 

procedure  developed  by  Brunk  (1955,  1958).  If  we  fix  t  ,  the  problem  is 
similar  to  the  one  discussed  by  Barlow  and  Scheuer  (1964)  and  by  Ayer, 
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Brunk,  Ewing,  Reid,  and  Silverman  (1955)-  For  i^L,...,k  obtain  the 

empirical  distribution  function  F^n  (t)  =  m^(t)/n^  ,  where  m^(t)  *  number 

1 

of  observations  among  not  exceeding  t  .  Then  the 

Brunk  procedure  yields  for  fixed  t  the  MLE 

^  m  (t)+. . .  +m  (t) 

(5.1)  F.  (t)  =  max  min  - - - - - -  ,  i=l,2,...,k  . 

s£L  r<±  nr  +***+s 


To  prove  that  each  F^(t)  is  monotonic  increasing  in  t  and  so 
qualifies  as  a  distribution  function,  we  simply  note  that  each  m^(t) 

A 

is  monotonic  increasing  in  t  ,  implying  by  (5.1)  that  F^(t)  is  also 

A 

increasing  in  t  .  Thus  F^(t)  ,  0  £  t  <  »  ,  is  the  MLE  of  F^(t)  , 


i=l , . . . ,k  . 

5.2  Conservative  Confidence  Bounds  on  Final  Reliability  Ordinarily, 
to  obtain  a  confidence  bound  on  a  parameter,  we  use  a  statistic  with 
known  distribution.  Unfortunately,  in  the  present  case  the  relevant 
distributions  are  unknown.  However,  we  can  still  obtain  a  conservative 
confidence  bound  using  the  following  general  theorem. 

Theorem  5.1  Let  (a)  Y  be  an  observation  on  a  random  variable 
(in  general  vector  valued)  having  distribution  function  G(£,9)  ,  0 

A 

a  one  dimensional  parameter,  (b)  ©(if)  be  a  one  dimensional  statistic 

A 

based  on  the  observed  vector  Y  ,  (c)  p(9(Y))  be  a  1-a  upper  confidence 
bound  on  0  ,  where  p(u)  is  a  decreasing  function,  (d)  X  be  an 
observation  on  a  random  variable  (in  general  vector  valued)  having 

A 

distribution  function  F(x,9)  ,  and  (e)  9(Y)  be  stochastically 

A  A  A 

£  9(X)  .  Then  P[p(©(X))  ^  0  |  F(x,0)]  ^  1-cr  j  i.s.,  p(0(X))  is  a 
conservative  1-a  upper  confidence  bound  on  9  ,  the  parameter  of  the 
distribution  F  . 
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u  be  the  inverse  of  the  function  p  .  (in  case  of  ambiguity,  let  u(x) 
be  the  largest  value  satisfying  p(u)  =  x).  The 

A  A 

1-a  =  P[p(©(y))  ^  9  |  G(-jr,0)3  =  P[9(y)  <£  u(9)]  ;  the  first  inequality 

follows  from  (c),  the  second  holds  since  u  is  the  inverse  function 
of  p  and  p  is  decreasing.  By  (e) 

P[9(Y)  £u(9)  |G(^0)]  £P[0(X)  £u(0)  |F(x,0)]  =  P[p(0(X))  ^9  |  F(x,©)3  ; 
the  last  equality  holds  since  u  is  the  inverse  of  p  and  p  is 
decreasing.  Combining  the  statements  above  we  obtain 
P[p(0(X))  ^9  |'F(x,0)3  ^1-a  . 


If  p  is  not  continuous  or  strictly  increasing,  the  same  result  may  be 
obtained  by  limiting  arguments.  |j 

Other  cases  of  interest  are  covered  in 

A 

Corollary  5-2  (l)  If  p  is  an  increasing  function  and  9(x)  is 

A 

stochastically  larger  than  0(Y),  then  the  same  conclusion  follows. 

(2)  If  p  is  a  decreasing  function,  p(o(Y))  is  a  1-a  lower 

A 

confidence  bound  on  the  parameter  0  of  G  ,  and  0  (X)  is  stochastically 

A  A 

larger  than  0(f)  ,  then  p(o(X))  is  a  conservative  1-a  lower  confidence 
bound  on  the  parameter  0  of  F  . 

A 

(3)  If  p  is  an  increasing  function,  p(©(Y)  is  a  1-a  lower 

A 

confidence  bound  on  the  parameter  9  of  G  ,  and  9(X)  is  stochastically 

A  A 

smaller  than  0(Y)  ,  then  p(0(x))  is  a  conservative  1-a  lower  confidence 
bound  on  the  parameter  0  of  F  . 

The  proof  in  each  c°-se  is  similar  to  that  of  Theorem  3.1* 

Returning  to  the  model  described  in  Section  5.1*  let  us  set  up  a 
conservative  confidence  bound  (a)  on  F^(t^)  ,  the  reliability  at  the  final 
stage  corresponding  to  fixed  tQ  and  (b)  on  the  entire  distribution 
function  F^  . 
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(a)  Let  X'  be  a  binomial  random  variable  representing  the  number 
k 

of  successes  in  n  =  Z  n  independent  trials,  where  the  probability 

1=1  1 

of  success  on  each  individual  trial  is  F  (t  )  •  Then  X*  is 

kiC  U 

stochastically  larger  than  X  =  Z  [n  -m  (t  )]  since  X'  is  the  sum  of 

i=l  1  1  0 

n  independent  binomial  random  variables  each  of  which  is  stochastically 
larger  than  the  n  corresponding  binomial  random  variables  comprising 
X  .  Thus  using  Corollary  5*2  (3)  we  may  obtain  a  conservative  1-a 
lower  confidence  bound  for  F  (t  )  as  follows: 

K  U 

x  ^  ini 

1.  Find  the  value  p(x)  satisfying  Z  (  )p  q  =  l-a  , where 
k  i=0  1 

q  =  1-p,  x  =  Z  (n  -m  (t  ) )  . 
i=l  _ 

2.  Then  PtF^Ct^)  ^>_  p(x)]  ^  l-a  ;  i.e.,  p(x)  is  a  conservative 
l-a  lower  confidence  bound  for  Fk(tQ)  ,  the  reliability  at  the  last  stage 
of  development. 


(b)  To  obtain  a  conservative  confidence  bound  on  the  entire 


distribution  function  F^  ,  we  first  show 

Lemma  3. 3  Let  X^,...,X^n  be  independent  observations  from 
F±  ,  i=l,...,k  ,  with  F  (t)  ^  ...  ^  Fk(t)  for  all  t  ,  and  F(t)  be 
the  empirical  distribution  formed  from  all  the  observations  3 

i=l,...,k  .  Let  X^,...,X^  ,  i=l,...,k  ,  be  independent  observations 

a  i 

from  Fk(t)  and  F'(t)  be  the  empirical  distribution  formed  from 


^il3 ’ "  3^in  3  i=*l,...,k  .  Then  given  any 


A  V4MV  WAVU  y 


P[F(t)  ^u(t),-co<  t  <  oo]^.p[F'(t)  ^u(t),-«><  t  <  00] 


Proof  First  assume  each  F^  continuous,  and  strictly  increasing. 
Define  X^  =  F"1  Fk(X^ )  ,  j=l, . . .  ;  i=l, . . .  ,k  . 

Then  the  set  of  random  variables  X^  ,  0=1,..., n^  ;  i=l,...,k, 
has  the  same  joint  distribution  as  the  set  X^,,  j=&,...,n^j  i=l,...,k  . 
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ij  > 


j— J-; 


^  ^  y  +  •  ^  y  1^  ^ 


Thus  for  each  t  ,  F"(t)  ^FT(t)  ,  where  F"(t)  is  the  empirical 

distribution  formed  from  ,  ,3=1,..., n^  ,  1=1,,.., k  .  The  conclusion 

of  Lemma  5-3  follows  from  a  generalization  of  the  Lemma  on  page  73  of 


Lehmann  (1959)* 

If  any  F  is  not  continuous  or  strict3.y  increasing,  the  same 
result  may  be  obtained  by  limiting  arguments.  || 

Now  we  may  form  a  conservative  1-a  upper  confidence  bound  on 

A 

the  entire  distribution  function  F,  as  follows.  Let  F(t)  be  the 

k 

empirical  distribution  formed  from  all  n  observations  X  ,  j=l,...,n  ; 

1 J  1 

i=d, .  ,.,k  .  From  Birnbaum  and  Tingey  (1951 ),  we  may  find  e  satisfying 

TaX 

p[Bn(t)  +  €  H(t),-  oo  <  t  <oo]  =  1-a  ,  where  H(t)  is  a  distribution 

function,  and  Hn(t)  is  the  corresponding  empirical  distribution  function 
based  on  a  sa^le  of  size  n  from  B(t)  .  The  value  Is  independent 

A 

of  H(t)  .  Then  we  may  claim  p[F(t)  +  6^  F^(t),  0  <1  t  <  °°]  1-a  . 

A 

That  is,  F(t)  +  is  a  conservative  1-a  upper  confidence  bound  on 

the  entire  distribution  function  F^(t )  .  This  is  an  immediate  consequence 
of  Lemma  5*3. 

In  a  similar  manner  we  may  obtain  MLE's  and  conservative  confidence 
bounds  in  a  variety  of  other  reliability  improvement  models,  A  forthcoming 
paper  by  Barlow,  Pros chan,  and  Scheuer  will  give  details. 
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